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Abstract: 
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This study investigated whether two different versions of an online professional devel- 
opment course produced different impacts on the intended outcomes of the course. 
Variations of an online course for middle school algebra teachers were created for two 
experimental conditions. One was an actively facilitated course with asynchronous peer 
interactions among participants. The second was a self-paced condition, in which neither 
active facilitation nor peer interactions were available. Both conditions showed signifi- 
cant impact on teachers’ mathematical understanding, pedagogical beliefs, and instruc- 
tional practices. Surprisingly, the positive outcomes were comparable for both conditions. 
Further research is needed to determine whether this finding is limited to self-selected 
teachers, the specifics of this online course, or other factors that limit generalizability. 
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Introduction 

There has been a dramatic increase in online professional development 
(OPD) programs for teachers across the country, with offerings from col- 
leges and universities, school districts, regional service providers, state 
departments of education, and other non-profit and for-profit providers 
(Dede, 2006; Galley, 2002; Keller, 2005). These programs have taken on a 
wide variety of models and formats as the number of online courses has 
expanded. For example, OPD opportunities today include single-session 
workshops, semester-long courses, and extended interactions with online 
peers or mentors over time; they may involve individual study programs 
or participation in online professional communities; they may also incor- 
porate a variety of communication methods and media, such as shared 
online workspaces, asynchronous discussion boards, webcasts, videocon- 
ferencing, and a mix of face-to-face and online interaction (Dede, 2006; 
Treacy, Kleiman, & Peterson, 2002). 

Prolific expansion in the number and types of OPD programs has 
raised concerns about the quality of existing offerings and strong interest 
in guidelines for the design of effective online courses for teachers (Keller, 
2005; National Staff Development Council, 2001). In a recent review of 
empirical research conducted over the past decade, Whitehouse, Breit, 
McCloskey, Ketelhut, & Dede (2006) found that past studies of OPD have 
focused on documenting the designs, methods of implementation, and 
correlates of OPD programs, but have not been designed to establish valid 
causal links between specific OPD designs and teacher and student out- 
comes. A majority of studies of online learning have been based on higher 
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education courses, where the most common finding has heen no signifi- 
cant difference in college-student learning between those in face-to-face 
and online courses (Phipps & Merisotis, 1999; Russell, 1999). 

Within the field of traditional face-to-face teacher professional devel- 
opment, researchers and practitioners have built upon a substantial body 
of research to agree on a consistent set of characteristics associated with 
effective design and practice (Sparks, 2002). Traditional single-day work- 
shops and courses that focus on abstract teaching principles are inade- 
quate, researchers argue; more powerful professional development involves 
ongoing, concrete classroom support that is integrated into teachers’ daily 
practice. Through activities such as regular peer coaching and shared lesson 
planning, teachers improve their professional knowledge and skills while 
addressing the specific learning needs of their students and the challenges 
posed by their specific school contexts (Darling-Hammond, 1997; Renyi, 
1996; Sparks, 2002; Stigler & Hiebert, 1999). Effective professional devel- 
opment focuses on the subjects that teachers teach, is aligned with local 
curriculum standards, deepens teachers’ content knowledge, and fosters 
active and inquiry-based forms of learning (Ball & Cohen, 1999; Loucks- 
Horsley, 1995; Scotchmer, McGrath, & Coder, 2005; Sparks, 2002; Willis, 
2002). At the same time, high-quality professional development targets a 
small set of student learning goals and is centered on improving student 
outcomes rather than meeting teacher interests (Loucks-Horsley, 1995; 
Sparks, 2002). 

Another key feature of effective face-to-face professional development 
is teacher collaboration. Teacher groups that form to exchange knowledge, 
advice, and support for improving classroom teaching and student learning 
have been given names such as learning communities, professional com- 
munities, communities of practice, and communities of inquiry (Barab, 
MaKinster, & Scheckler, 2003; Dalgarno & Colgan, 2007; DuFour, 2002; 
Lim, 2004; Louis & Marks, 1998; Nickerson 8t Sowder, 2002). Researchers 
argue that membership in collaborative learning communities is critical 
for teacher professional development, because ongoing discussion with 
peers encourages analytical and reflective thinking and helps to spread the 
“wisdom of practice” (Ball 8t Cohen, 1999; Schulman, 1987). When teachers 
develop open and trusting relationships with community colleagues, they 
begin to feel less isolated, more willing to work together to tackle student 
issues, and more open to experiment and to take risks with methods of 
instruction (Dalgarno 8t Colgan, 2007; DuFour, 2002). Some researchers 
have found statistically significant relationships between the presence of 
professional communities and higher student achievement, arguing that 
strong teacher collaboration supports the use of pedagogical methods that 
can promote deeper student learning (Louis 8c Marks, 1998). 
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While the research on OPD is far less extensive, OPD programs can 
seek to incorporate characteristics associated with effective face-to-face 
professional development while offering a number of potential advantages 
over face-to-face formats. Because Internet access has become nearly uni- 
versal in K-12 schools and widespread in teachers’ homes, OPD provides 
teachers with opportunities to engage in forms of training that may not be 
available within their local areas. Teachers can participate in professional 
development during times that are convenient; receive job-embedded 
support that addresses immediate classroom needs; customize programs 
to better suit their own individual learning styles; interact with material 
through a variety of visual or other multi-media formats; and gain valu- 
able computer and online technology skills (Docherty & Sandhu, 2006; 
Garrison & Cleveland-Innes, 2005; National Staff Development Council, 
2001; Richardson, 2002; Spicer, 2002; Treacy, Kleiman, & Peterson, 
2002 ). 

Due to its accessible and flexible modes of communication, OPD can 
also support the growth and maintenance of teacher learning communi- 
ties. The internet can connect teachers to broad networks of professionals, 
providing them with access to a wide array of learning experiences, and 
helping to reduce the isolation that has often accompanied traditional 
forms of classroom teaching (DuFour, 2002; National Staff Development 
Council, 2001). Some researchers and educators argue that OPD can pro- 
mote deeper levels of communication and thought among teachers than 
face-to-face forms of professional development. Because online programs 
can store written records of teacher conversations, and because teachers 
can participate in group discussion asynchronously, OPD allows teachers 
to contribute ideas when they are ready and to be more reflective in their 
written, online comments (Spicer, 2002; Treacy, Kleiman, & Peterson, 
2002). For sensitive topics, online discussion boards can sometimes foster 
more open and uninhibited dialogue (Spicer, 2002). By providing teachers 
with the ongoing ability to communicate online, OPD can strengthen 
professional relationships that are created face-to-face and can sup- 
port ongoing teacher collaboration after a course ends (Anderson, 2002; 
Dalgarno & Colgan, 2007; Garrison & Cleveland-Innes, 2005; Treacy, 
Kleiman, & Peterson, 2002). 

While OPD has the potential to facilitate collaboration, there are also 
many challenges in fostering successful online collaborations. Researchers 
argue that truly collaborative teacher cultures and sustainable learning 
communities cannot be designed, created, or controlled through outside 
forces; they must be member-driven, involve voluntary participation, 
and evolve to meet the needs of members over time (Barab, MaKinster, 
& Scheckler, 2003; Hargreaves, 1994; Henderson, 2007). Research has 
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shown that building learning communities online can take long periods 
of time and require significant energy and input from course designers 
and facilitators (Hur & Hara, 2007; him, 2004; Schlager, Fusco, 8t Schank, 
2002). Access to varied forms of communication and a high quantity of 
teacher interaction is insufficient for the development of effective learning 
communities. Instead, designers of professional development must pro- 
mote teacher interaction that is structured, purposeful, substantive, and 
engaging in order to deliver high quality support and stimulate deeper 
professional growth (Ball, 2002; Garrison & Cleveland-Innes, 2005). 

Researchers and practitioners therefore suggest that for OPD to reach 
its full potential, programs must offer content and skills training that 
are directly relevant to teachers’ immediate classroom needs. They must 
contain a variety of activities to engage learners; allow asynchronous 
online discussion to allow for varied schedules; provide multiple oppor- 
tunities for teacher reflection; employ skilled online course facilitators; 
and give teachers the ability to continue collaboration after the course 
ends (Anderson, 2002; Dalgarno & Colgan, 2007; Florida Instructional 
Technology Resource Center at UCF, 2000; him, 2004). To build sustain- 
able online learning communities, researchers suggest that designers work 
with teachers to co-develop course content and topics of discussion (Barab, 
MaKinster, & Scheckler, 2003; Dalgarno & Colgan, 2007). Researchers 
have found close emotional and personal connections among members 
of enduring professional communities. To help build such connections, 
some writers suggest that OPD developers incorporate face-to-face ses- 
sions within their courses and provide online discussion spaces in which 
teachers can interact more socially and informally (Florida Instructional 
Technology Resource Center at UCF, 2000; Galley, 2002; Treacy, Kleiman, 
& Peterson, 2002). 

These recommendations emerge from observations of existing OPD 
programs, surveys gauging the reactions of participants to specific online 
learning experiences, conceptual comparisons of online and face-to-face 
learning, and generalizations of principles of effective professional devel- 
opment in face-to-face contexts. To date, we have been unable to identify 
any research that examines the relationship between specific features of a 
course that are manipulated and learning outcomes measured by a formal 
test administered to participants or their students. While these sugges- 
tions may be useful to those designing and implementing OPD programs, 
they are not based on research that empirically demonstrates the effects of 
specific design features on teacher and student outcomes. Currently, many 
key questions about how to structure and implement OPD to improve 
teacher knowledge and practices remain unanswered. To help build this 
knowledge base, researchers at Education Development Center, Inc (EDC) 
and the Technology and Assessment Study Collaborative at Boston College 
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collaborated to conduct a series of studies that focused on how the amount 
of interaction among participants, the amount and type of facilitation, 
and the medium of the course (i.e., online or face-to-face) affected the 
intended outcomes of a given course. This paper presents findings from 
the first of three studies and was designed specifically to address the fol- 
lowing question: 

In an online professional development course designed to 
improve teachers’ content knowledge and abilities to teach 
algebraic thinking in the middle grades, do teachers who 
participate in a facilitated online course that includes interacting 
within a cohort of participants in weekly sessions have different 
learning outcomes than teachers who work through an online 
course individually and at their own pace? 

An experimental study was conducted to examine how the two dif- 
ferent approaches to OPD may affect the goals or intended outcomes of the 
course, namely increasing teacher content knowledge, altering teachers’ 
pedagogical beliefs, and changing teachers’ instructional practices. Thus, 
the theoretical framework guiding this research holds that: a) online pro- 
fessional development can affect teachers’ content knowledge, pedagog- 
ical beliefs, and instructional practices; and b) the outcomes of an online 
professional development course are dependent, in part, upon the level of 
facilitation and interaction among participants. Based on generally recom- 
mended practices for OPD, it was hypothesized that the Facilitated Cohort 
course would have greater impact than the Self-Paced course on teacher 
knowledge acquisition and classroom practices. 
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The Course: Building Algebraic Thinking in 
the Middle Grades 

The course employed for this study was titled Building Algebraic Thinking 
in the Middle Grades and was based on a book authored by Mark Driscoll 
titled Fostering Algebraic Thinking (Driscoll, 1999). The course was devel- 
oped by mathematics experts at EDC and focused on three important alge- 
braic concepts: patterns, functions, and number sense. 

For the study presented here, two versions of the course were pro- 
vided. Both versions required teachers to read the same material, conduct 
the same activities in their classroom, and complete the same assignments 
within a ten-week period. In addition, the courses were run by the same 
set of two facilitators, each facilitating one session of each version of the 
course. The intended learning outcomes of the course were also the same 
for both versions. These outcomes included the following: 

• Increased understanding of the concepts of patterns, functions, 
and number sense; 

• Increased ability to identify misconceptions and errors that 
students apply when working on problems specific to patterns, 
functions, and number sense; 

• Increased ability to identify and apply instructional strategies 
that hold potential to help improve student understanding of 
these three concepts; 

• Increased use of instructional methods that allow students to 
develop their own thinking, communicate their understandings, 
and respond to their classmates’ problem solving strategies. 

The course activities were divided into 10 sessions, preceded by a brief 
orientation pre-session to familiarize participants with the technology 
and course expectations. Sessions contained readings, videos, mathemat- 
ical activities for the teacher participants, and pedagogical activities, such 
as preparing a lesson or interviewing students about their mathematics 
thinking. Depending upon the version of the course, session requirements 
also included participating in an asynchronous online discussion seeded 
by specific questions or completing an individual journal entry addressing 
the same questions. Participants were expected to spend about 4 hours 
per session, and to complete a final project. The course content is summa- 
rized in Table 1 (next page) and the full course is available at: http://www. 
curriki.org/xwiki/bin/view/Coll_edcl/BuildingAlgebraicThinkingthroug 
hPatternFunctionandNumber-ProfessionalDevelopmentCourseforMiddle 
GradeMathTeachers. 
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Table 1 : Building Algebraic Thinking in the Middle Grades through Patterns, 

Functions, and Number Sense 


Pre-session: 

Orientation 

You will become familiar with the course format, the facilitators' 
backgrounds, and what is expected of participants. You will also perform 
a technology check on your computer to make sure you 
have the required software. 

Session 1: 

What is Algebraic 
Thinking? 

In this introductory session, you will learn about different definitions 
of algebraic thinking including the definition used in this course, and 
compare those definitions to your own. You will also do several math 
problems that will further clarify the concept of algebraic thinking, 
and participate in the online discussion on defining algebraic thinking. 

Session 2: 

Exploring Algebraic 
Thinking in Patterns 

In this session, you will focus on your own mathematical thinking when 
doing pattern problems. An understanding of your thinking will serve as 
a starting point when you examine student thinking in later sessions. 

Session 3: 

Analyzing Students' 
Algebraic Thinking about 
Patterns 

This session explores ways to understand students'algebraic thinking by 
examining their work and discussion in the context of pattern activities. 

Session 4: 

Using Teacher Question- 
ing to Develop Algebraic 
Thinking 

This session is about developing good questioning techniques that help 
students develop algebraic thinking in the context of pattern activities. 

Session 5: 

Exploring Algebraic 
Thinking with Functions - 
Making Generalizations 

This session explores how to foster students'algebraic thinking habits 
as they build generalization skills which are essential to understanding 
functions. 

Session 6: 

Exploring Algebraic 
Thinking with Functions - 
Understanding Graphs 

This session explores ways to foster students'algebraic thinking habits 
as they learn to analyze and interpret graphs of functions. 

Session 7: 

Using Interviews to Ana- 
lyze Students'Thinking 
about Functions 

This session explores how student interviews are an effective tool for 
learning about students'algebraic thinking. 

Session 8: 

Exploring Algebraic 
Thinking with Number 
Sense - Algorithmic 
Thinking 

This session examines how number strand activities can be used to 
develop students'algorithmic thinking, an aspect of algebraic thinking. 

Session 9: 

Exploring Algebraic 
Thinking with Number 
Sense - Developing 
Symbol Sense 

This session examines how number strand activities can be used to 
develop students' symbol sense, an aspect of algebraic thinking. 

Session 10: 
Post-Course Survey 
and Wrap-up 



Online Courses for Math Teachers 


Carey, Kleiman, Russell, Venable, & Louie 


n 


The two versions of the course differed in the role of the facilitator, 
whether interactions occurred among participants, and the schedule at 
which participants worked through the course material. In the Facilitated 
Cohort version, participants worked through course material together at 
the same pace and covered one session of the course per week. In addi- 
tion, these participants were strongly encouraged to interact with their 
colleagues, via an asynchronous, text-based discussion forum, by posting 
responses to questions raised by the facilitator and also responding to 
other participants’ comments. For each session of the course, each partici- 
pant was required to make a minimum of two contributions to the discus- 
sion forum. In this version of the course, the facilitator participated in the 
online discussion, seeking to engage all participants in it, and provided 
extended feedback to participants nearly every week, and sometimes more 
often depending upon a participant’s learning needs. Thus, the Facilitated 
Cohort was designed to: a) strongly encourage interactions among par- 
ticipants throughout the course; b) control the pace at which participants 
worked through the course material; and c) maximize the role of the facili- 
tator in interacting with individual participants and in guiding the devel- 
opment of the cohort as a group. 

In the Self-Paced version, participants worked through the course 
material at their own pace, and were only required to complete all written 
assignments within a 10 week period. Participants in the Self-Paced course 
did not have access to a discussion board or any way to contact other 
course participants. The only communication participants had was with 
the course facilitator, who provided feedback via email on the assignments 
completed by participants and did not engage in interactions with partici- 
pants other than clarifying questions about a given assignment. Thus, the 
Self-Paced versions was designed to: a) prevent interactions among course 
participants; b) allow participants to work through the course mate- 
rial at their own pace; and c) minimize the role of the facilitator to only 
responding to direct questions about assignments and providing feedback 
on assignments directly to each participant. The study, then, was designed 
to examine whether learning outcomes, defined by the learning goals for 
the course, differed when the course was led by a facilitator who played 
an active role in guiding discussions and strongly encouraged interac- 
tions among participants versus a course that was led by a facilitator who 
responded only to individual questions and assignments and did not allow 
any interactions among participants. 
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Study Design 

The study presented here was designed to compare the learning out- 
comes of the same course content delivered as either a Facilitated Cohort 
or a Self-Paced only course. Two “classes” of each version were provided, 
with each intended to have up to 25 teachers participate. Two facilitators 
were hired to run the courses, each with experience providing professional 
development sessions and training in online teaching. To control for the 
effect that an individual facilitator may have on the learning outcomes of 
the course, each facilitator was responsible for one class of each type of 
course. 

Participation in the study was limited to middle school teachers who 
were currently teaching at least one algebra course. Messages inviting 
teachers to participate in the study were distributed via a variety of list- 
servs that focus on mathematics instruction or were run by state orga- 
nizations with whom the researchers had prior working relationships. 
Ninety-seven public school teachers who responded to the invitation, were 
currently teaching math to 7* or 8* grade students, and had a working 
email address were selected to participate in the study. Teachers were then 
stratified by gender. Forty-eight teachers were randomly assigned to the 
Self-Paced treatment group and 49 were assigned to the Facilitated Cohort 
treatment group. Each treatment group was then randomly split again 
into two classes in order to limit the number of participants in a given 
class to 25 or fewer. 

Participants who completed the course and the data collection instru- 
ments were awarded either 5 quarter or 3 semester hours of graduate 
course credit or a stipend of $200. As described in greater detail below, 
of the 91 teachers who began the study, 52 met all of the course and data 
collection requirements. 
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Data Collection Instruments 

Six instruments were employed to collect data about the intended out- 
comes of the online courses: (1) a background survey, (2) a pedagogical 
beliefs and practices survey, (3) a measure of teachers’ understanding of 
teaching algebraic concepts, (4) a student survey, (5) a teacher log, and (6) 
a course evaluation. As shown in Table 2, the background survey, peda- 
gogy survey, teacher log, and the mathematics assessment were adminis- 
tered during a 5 day period prior to the start of the course. The pedagogy 
survey, mathematics assessment student survey, teacher logs, and course 
evaluation were administered during a 10 day period following the end 
of the course. Each of the instruments is described briefly below and the 
instruments are available online at http://www.bc.edu/research/intasc/ 
researchproj ects/ optimizingOPD/ O PD. shtml. 

Table 2: Overview of Instrument Administration 


Instrument 

Administered 

Pre-course 

Administered 

Post-course 

Background Survey 

X 


Pedagogy Survey 

X 

X 

Math Assessment 

X 

X 

Student Survey 

X 

X 

Teacher Log 

X 

X 

Course Evaluation 


X 


Background Survey 

The Background Survey was designed to collect demographic infor- 
mation and information about teachers’ prior experiences with profes- 
sional development and technology use. The majority of the items on the 
Background Survey were closed-response. In a few instances, participants 
were requested to type in numeric values. A second set of questions were 
developed around seven categories. Scales were developed based on theory 
and were confirmed through principal components analyses. These seven 
scales and their associated reliabilities included: 1. Confidence in Teaching 
Math (.85), 2. Instructional Practice (.87), 3. Teaching Experience (.60), 4. 
Professional Workshops (.56), 5. Teachers’ Access to Technology at Home 
(.79), 6. Teachers’ Access to Technology at School (.78), and 7. Technology 
Use (.78). 
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The Pedagogy Survey 

The Pedagogy Survey collected information about teachers’ pedagog- 
ical beliefs and instructional practices. All items were closed-ended and 
asked teachers to either report the frequency with which they employed 
specific instructional techniques or to indicate the degree to which they 
agreed or disagreed with a statement that focused on the value of a given 
instructional practice. The vast majority of items employed for this survey 
were taken from two previous studies that focused on the relationships 
between pedagogical beliefs and practices and the use of instructional 
technology in the classroom (Becker, 1999; Russell, O’Dwyer, Bebell, & 
Miranda, 2004). The specific scales that were formed and the associated 
reliabilities included: 1. Teacher Centered Beliefs (.74); 2. Student Centered 
Beliefs (.66); 3. Instructional Use of Technology (.77); 4. Discussing and 
Demonstration Solutions (.74); 5. Assigning Higher Order Problems (.58); 
6. Assigning Projects (.65); 7. Assigning Worksheets (.60); and 8. Writing 
About Mathematics (.61). 

Math Assessment 

The Math Assessment was designed to collect information about 
teachers’ understanding of mathematical concepts covered in the profes- 
sional development course. The assessment was administered twice, once 
in the first week and once in the final week. The assessment presented 
teachers with a sample of student work for a given problem related to pat- 
terns, functions, or number sense. The teacher was then asked to respond 
to a series of questions about the student’s work. For each sample, teachers 
were asked to: a) identify the content and process goals measured by the 
task the student responded to; b) assess whether the student appears to 
understand the mathematics required for the problem; c) whether any mis- 
conceptions were present in the student’s work; d) what more the teacher 
would like to know about the student’s understanding based on the work 
sample; and e) how the teacher should deconstruct the students’ under- 
standing. Using a scoring guide that was shown to yield reliable scores 
(exact agreement exceeded 85% for all items), two readers independently 
scored teacher responses and, when discrepancies occurred, the readers 
discussed their scores before reaching a consensus score. The scoring guide 
employed the following four point scale: (1) does not meet expectations; 
(2) partially meets expectations; (3) meets expectations; and (4) exceeds 
expectations. 
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Teacher Logs 

The Teacher Logs were designed to capture information about teachers’ 
day-to-daypedagogical practices and were intended to document the extent 
to which teachers employed specific instructional strategies during a given 
lesson (see teacher log for the specific instructional strategies for which 
data were collected). The Teacher Logs were administered twice, once in 
the first week of the course and once in final week of the course. During 
each administration, teachers were asked to select one class in which they 
teach algebra and to then complete a log for each of five times the teacher 
taught that class. Each log consisted of a series of instructional strategies 
similar to those explored in the course. For each strategy, teachers were 
asked to indicate whether the strategy was: (1) not used at all; (2) a minor 
component of the lesson; (3) a major component of the lesson; or (4) the 
most important component of the lesson. The ratings provided for each 
activity were then averaged across the logs recorded for each week. 

Student Survey 

To help triangulate data provided by teachers via the pedagogy survey 
and the teacher logs, a survey was administered on paper to students in the 
teacher’s algebra classroom. The survey’s items asked students to indicate 
the frequency with which they engaged in specific learning activities (e.g., 
performing worksheets individually, working with partners to solve prob- 
lems, sharing solutions with their class, etc.) and how often their teacher 
employed specific instructional strategies (e.g., asking students to explain 
how they solved a problem, showing students how to solve a problem, 
asking students to respond to each others work, etc.). Teachers were asked 
to administer the survey to the same class selected for the teacher log. 
Students completed the survey once prior to the course and once following 
the course. All items were forced-choice and were used to form scales that 
represent the degree to which students engaged in student-centered activ- 
ities and in teacher-directed activities. The scale employed for these items 
ranged from: (1) almost always; (2) most of the time; (3) once in a while; 
and (4) never. Hence, a low score for a given item indicated more frequent 
use of the given instructional strategy. 

Course Evaluation 

The Course Evaluation was designed to collect information from 
teachers about the positive and negative aspects of the course. The course 
evaluation instrument asked participants to rate the value and quality of 
various aspects of the course including the reading material, the assign- 
ments, the facilitator, feedback on assignments, etc. (see the course evalu- 
ation instrument to view all elements for which data was collected). The 
Course Evaluation was only administered as a post measure. 


I-T-L-A 


Online Courses for Math Teachers 


Carey, Kleiman, Russell, Venable, & Louie 


16 


Findings 

The primary research question examined in this study asked whether 
the combination of altering the pace with which teachers performed course 
material and the level of interaction with their peers and the facilitator 
affected teachers’ mathematical understanding, their pedagogical beliefs, 
and their instructional practices. To this end, the majority of analyses con- 
ducted for this study focused on comparing the effect of the two versions 
of the course on these three outcomes. In addition, to provide a better 
understanding of characteristics of the study participants, descriptive sta- 
tistics were calculated for several items on the background survey. Finally, 
since completion of a professional development course/program may be 
affected by both the mode of delivery and the characteristics of the pro- 
gram itself (e.g., self-paced versus group facilitation), separate analyses 
were conducted to examine the effect that the two forms of the course had 
on attrition. We begin with results of the background survey. 

Characteristics of Study Participants 

Within each treatment group, the majority of teachers were female 
(79% for the Self-Paced and 71% for the Facilitated Cohort group). 
Fifty-one percent of teachers in each treatment group categorized them- 
selves as between the ages of 45 to 59. The Self-Paced treatment group 
had slightly more teachers in their thirties (26% versus 16%). Thirty-two 
percent of teachers from the Self-Paced and 33% of teachers from the 
Facilitated Cohort group indicated that they had been teaching between 1 
and 5 years. Nineteen percent of the Self-Paced and 13% of the Facilitated 
Cohort group indicated they had been teaching for 6 to 10 years. The 
Self-Paced group had 17% and the Facilitated Cohort group had 16% par- 
ticipants teaching for between 11 and 15 years. Thirty percent of partic- 
ipants in the Self-Paced group taught for 16 years or more and 34% of 
teachers in the Facilitated Cohort group taught for the same time range. 
Approximately 2% of participants from Self-Paced and 4% of participants 
from Facilitated Cohort had taught for less than 1 year at the time of the 
online course. 

Participants in both groups reported similar educational backgrounds 
and certification status. Twenty-eight percent of Self-Paced and 24% of 
Facilitated Cohort participants majored in mathematics as undergrad- 
uate students. Twenty-one percent of Self-Paced participants and 20% of 
Facilitated Cohort participants minored in mathematics. In both groups, 
less than half of the participants had a master’s degree. Further, most par- 
ticipates did not have certification in the state in which they taught. Only 
26% of Self-Paced participants and 31% of Facilitated Cohort participants 
were state certified. 
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A t-test was conducted on each of the seven scales to examine whether 
the treatment groups differed with respect to their confidence in teaching 
math, instructional practices, teaching experience, professional develop- 
ment experiences, access to technology at home and at school, and their 
use of technology. No statistically significant differences between the 
groups for any of these background variables were detected. 

Completion Rate 

Of the 97 participants who agreed to participate in the study, four 
did not complete the pre-course background instrument or the other 
pre-course measures. For this reason, they were dropped from the study 
before the professional development course began. During the study, an 
additional 39 (44%) teachers either did not complete the course or did not 
complete the required post-course data collection instruments resulting in 
a total of 53 participants. Analysis of those who dropped out of the study 
indicates that a larger number of participants dropped out of the learning 
communities group (26) than the Self-Paced group (18). The characteris- 
tics of those who dropped out did not differ between the two treatment 
groups. Follow-up surveys with the drop outs indicated that a large per- 
centage reported they were unable to complete the course due to family 
health issues, while a smaller number of drop-outs indicated that they 
found the time requirements of the course too demanding. 

The majority of drop-outs were between the ages of 35 and 50. In addi- 
tion to this age trend, chi-square analyses revealed three other character- 
istics of teachers who tended to persist versus those who dropped out. 
First, teachers who either minored or majored in math were more likely 
to persist through the course. Second, teachers who were certified in math 
in the state in which they teach were more likely to persist through the 
course. Finally, teachers who use computers daily or almost daily were 
more likely to persist through the course. Our interpretation is that this 
was a demanding course with significant mathematical content, and that 
teachers who had strong mathematics backgrounds were therefore more 
likely to complete the course. 

Quality of the Online Course 

Overall, participants believed that both versions of the course were 
well developed, well delivered, and valuable. Across all items on the survey 
that asked participants to rate the value of specific components of the 
course and the course overall, the mean fell between the valuable and very 
valuable options. As just one example, when asked to “rate your learning 
in this course compared to university or college courses you have taken 
on campus” the mean rating was 3.13, which fell between “slightly more 
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valuable (3)” and “much more valuable (4).” This mean rating indicates 
that participants generally felt they learned as much or more in the 10 
week online professional development course than they did in their prior 
university-sponsored courses. 

To examine whether participants in the Self-Paced and the Facilitated 
Cohort groups had similar views about the course, a series of t-tests 
were conducted for each item on the end-of-course evaluation. Without 
adjusting for multiple comparisons, a statistically significant difference 
was found for only one item. This item asked participants, “How valuable 
did you find reflecting in learning logs?” The mean for the Self-Paced group 
was 3.43, which indicated that the group valued the learning logs. The 
mean for the Facilitated Cohort group was 3.04, which indicated less value 
for the learning logs. When adjusted for multiple-comparisons, this dif- 
ference was no longer significant. Collectively, the lack of significant dif- 
ference between the two groups ratings of the course indicate that the 
form of the course (i.e., Self-Paced versus Facilitated Cohort) did not seem 
to influence the participants views of the quality or utility of the course. 
Overall, the course was highly rated on all aspects by participants in both 
versions. 

Changes in Pedagogical Beliefs 

The Pedagogy Survey was designed to collect information about 
teachers’ pedagogical beliefs and practices. The survey was administered 
pre-and post-course, and the data was used to examine changes in teachers’ 
beliefs and practices that followed their participation in the course. It 
should be noted that the scales were standardized to have a mean of 0 
and a standard deviation of 1. Table 3 (next page) displays the mean scale 
scores for each treatment group for the pre- and post-course adminis- 
trations. The column labeled Post-Pre displays the change in mean score 
between the pre-course and post-course administrations. For both treat- 
ment groups, participation in the course appeared to have the strongest 
effect on teacher-directed beliefs, leading to a large decrease in the strength 
of those beliefs. Interestingly, the course also appeared to decrease the 
strength of student-centered beliefs. Together, these changes suggest that 
the course helped teachers develop more balanced beliefs about the value 
of teacher-directed and student-centered instructional strategies. 

In addition. Table 3 indicates that the course had a slight negative effect 
on the strength of beliefs about the teacher discussing and demonstrating 
solutions. Conversely, teachers developed stronger beliefs about the value 
of assigning higher-order problems and writing about mathematics. The 
effects for the other variables were inconsistent across the two treat- 
ment groups. Of most interest, however, is that the change in teachers’ 
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beliefs differed significantly across the two treatments for only one vari- 
able, namely student-centered beliefs. As seen in Table 3, teachers in the 
Facilitated Cohort group experienced a relatively large change in their 
beliefs while the change for the Self-Paced group was more moderate. 

Table 3: Scale Score Difference Between Pre- and Post-Survey, 

Grouped by Treatments 




Self-pace 

Facilitated Cohort 



N=30 


N=23 


Pre 

Post 

Post - Pre 

Pre 

Post 

Post - Pre 

Teacher Directed Beliefs 
Scale 

0.65 

-0.59 

-1.24 

0.62 

-0.70 

-1.32 

Student Centered Beliefs 
Scale* 

0.14 

-0.06 

-0.20 

0.25 

-0.36 

-0.61 

Technology Use Scale 

-0.17 

-0.12 

0.05 

0.28 

0.11 

-0.19 

Discussing or 
Demonstrating Solutions 
Scale 

0.06 

-0.18 

-0.23 

0.24 

-0.08 

-0.32 

Assigning Higher Order 
Problems Scale 

-0.03 

0.13 

0.15 

-0.16 

0.02 

0.18 

Assigning Projects Scale 

0.11 

0.08 

-0.03 

-0.24 

-0.01 

0.23 

Assigning Seat or 
Homework Scale 

0.03 

0.21 

0.18 

-0.14 

-0.17 

-0.02 

Writing about Math Scale 

0.02 

0.25 

0.23 

-0.24 

-0.12 

0.13 


* Change scores differed significantly between groups at p = 0.05. 


Changes in Instructional Practices 

Teachers completed five teacher logs prior to the course and five logs 
following the completion of the course. As described above, the logs asked 
teachers to indicate the extent to which each type of activity was a com- 
ponent of the logged lesson. Table 4 (next page) displays the mean rat- 
ings given by teachers across lessons logged prior to and following the 
course. The column labeled post-pre indicates the change in the mean 
rating between the pre-course logs and the post-course logs. Within the 
Self-Paced group, the extent to which instructional strategies were ele- 
ments of lessons changed significantly for six strategies. Three of these 
strategies - doing introductory drills, working with manipulatives, and 
working on independent long-term projects - saw declines in their impor- 
tance. Conversely, three strategies - presenting an answer to a problem 
in words, debating ideas or explaining reasoning, and working together 
on problems - experienced increases. For the Facilitated Cohort group, 
significant changes were found for six strategies. Specifically, four strat- 
egies experienced declines - doing introductory drills, working with 
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manipulatives, working on independent long-term projects, and admin- 
istering multiple-choice quiz/tests. Four strategies experienced increases 
- demonstrating/explaining concepts to the whole class, debating ideas or 
explaining reasoning, working together on problems, and having students 
respond to each other. Comparisons of change across the two treatment 
groups revealed no significant differences, suggesting that changes in the 
extent to which teachers reported the use of each instructional strategy 
were similar across the two treatments. 

Table 4: Summary of Teacher Log Mean Scores by Treatment Group 



Self-paced 

Learning Community 

Pre 

Post 

Post - Pre 

P 

Pre 

Post 

Post - Pre 

P 

1 

1 had the students do 
introductory drills 

2.23 

1.47 

-0.76 

<0.01 

2.09 

1.56 

-0.53 

<0.01 

2 

1 asked the students to 
make conjectures 

2.19 

2.14 

-0.05 

0.74 

1.93 

2.16 

0.22 

0.11 

3 

1 asked the students to 
present an answer to a 
problem visually 

2.37 

2.17 

-0.20 

0.21 

2.33 

2.22 

-0.01 

0.56 

4 

1 asked students to 
present an answer to a 
problem with algebraic 
symbols 

2.35 

2.31 

-0.04 

0.85 

2.27 

2.37 

0.10 

0.64 

5 

1 asked students to 
present an answer to a 
problem in words 

1.80 

2.22 

0.42 

0.04 

1.88 

2.15 

0.27 

0.11 

6 

1 had students work on or 
review homework 

1.60 

1.67 

0.09 

0.67 

1.66 

1.76 

0.13 

0.43 

7 

1 worked with/ 
demonstrated with 
manipulatives (i.e. 
algebra blocks) 

2.34 

1.49 

-0.85 

<0.01 

2.43 

1.82 

-0.61 

0.01 

8 

1 asked follow-up 
questions to students' 
responses to questions 

2.17 

2.36 

0.19 

0.21 

2.18 

2.45 

0.27 

0.06 

9 

1 led whole class 
discussions 

2.23 

2.06 

-0.17 

0.23 

2.26 

2.11 

-0.15 

0.37 

10 

1 demonstrated/ 
explained concepts to 
whole class 

1.92 

2.02 

0.10 

0.40 

1.67 

2.06 

0.40 

0.02 

11 

1 addressed routine 
or textbook-based 
problems 

1.76 

1.62 

-0.14 

0.40 

1.44 

1.60 

0.16 

0.24 

12 

1 used worksheets 

1.41 

1.50 

0.08 

0.67 

1.26 

1.51 

0.25 

0.06 

13 

1 had students work on 
independent, long-term 
projects 

1.89 

1.45 

-0.44 

0.01 

1.97 

1.28 

-0.69 

<0.01 
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Table 4: Summary of Teacher Log Mean Scores by Treatment Group 

(continued) 



Self-paced 

Learning Community 

Pre 

Post 

Post - Pre 

P 

Pre 

Post 

Post - Pre 

P 

14 

1 had students 
develop technical or 
mathematical writing 
skills, including 
equations, graphs, and 
tables. 

2.37 

2.03 

-0.34 

0.10 

2.02 

1.78 

-0.24 

0.28 

15 

1 asked students to solve 
real-world problems 

2.28 

2.16 

-0.13 

0.51 

2.29 

2.01 

-0.28 

0.06 

16 

1 had students work in 
pairs or small groups 

2.05 

2.23 

0.18 

0.24 

2.06 

2.29 

0.23 

0.34 

17 

1 had students work 
individually 

2.19 

2.03 

-0.16 

0.39 

2.26 

1.99 

-0.27 

0.25 

18 

1 had students respond 
to one another 

2.13 

2.21 

0.08 

0.55 

2.06 

2.42 

0.36 

0.02 

19 

1 had students debate 
ideas or otherwise 
explain their reasoning 

1.55 

2.24 

0.69 

<0.01 

1.68 

2.30 

0.62 

<0.01 

20 

1 had students work 
together on problems 
for which there is no 
immediately obvious 
method or solution 

1.32 

1.77 

0.45 

0.03 

1.03 

1.82 

0.78 

<0.01 

21 

1 administered a multiple 
choice test/quiz 

1.32 

1.23 

-0.09 

0.46 

1.36 

1.06 

-0.30 

<0.01 

22 

1 administered an open- 
ended test/quiz 

1.44 

1.23 

-0.21 

0.09 

1.23 

1.29 

0.06 

0.60 


Student Survey Results 

Similar to the teacher logs, analyses of the student survey focused on 
responses to individual items. For each item, the mean response was calcu- 
lated for each treatment group (Self-Paced and Facilitated Cohort) during 
each administration period (pre-course and post-course). Recall that the 
scale was structured such that a low score (1) represented frequent use 
of the strategy while a high score (4) indicated that the student reported 
never being exposed to the strategy. Table 5 displays the change in mean 
response within each group. Within the Self-Paced group, four items saw 
significant changes in the extent to which students reported their teacher 
employing a given practice. Three of these practices - explain how 1 got 
my answers, using calculators, and using computers - saw significant 
increases in use. The remaining practice - copying notes from the board 
- experienced a significant decrease. For the Facilitated Cohort group, sig- 
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nificant changes were also detected for four items. Three of the instruc- 
tional practices - explaining how I got my answers, using physical objects, 
and working on problems that take an entire class to finish - saw signifi- 
cant increases in use. Copying notes from the board saw decreased use. 

Examining changes between the students of teachers in the Self-Paced 
versus the Facilitated Cohort versions of the course revealed two items that 
differed significantly. For item 14, which focused on the use of worksheets 
that contain short problems, students in the Self-Paced treatment group 
reported that their teachers gave worksheets less frequently in the post- 
survey than in the pre-survey. Students of Facilitated Cohort teachers, on 
the other hand, reported that teachers gave worksheets more frequently 
in the post-survey than they reported in the pre-Student Survey. A differ- 
ence between treatment groups was also found for Item 15, which focused 
on the use of extended problems. Students of participants in the Self- 
Paced group moved slightly toward the response once in a while. Students 
of participants in the Facilitated Cohort group moved slightly toward the 
response, most of the time. 

Table 5: Mean Score Difference Between Pre- and Post-Surveys, by 

Treatment Groups 



Self -paced Cohort 
(25 classrooms, 447 students) 

Facilitated Cohort 
(21 classrooms, 386 students) 

Pre 

Post 

Post - Pre 


Pre 

Post 

Post - Pre 

V 

1 

My teacher asks me 
to explain how 1 got 
my answers to math 
problem 

1.75 

1.59 

-0.16 

<0.01 

1.79 

1.69 

-0.10 

0.05 

2 

In my math class, my 
teacher tells us what 
we need to do to get 
a good grade on an 
assignment 

1.60 

1.65 

0.05 

0.35 

1.59 

1.64 

0.05 

0.37 

3 

In my math class, we 
practice things over 
and over until we get 
them right 

1.83 

1.86 

0.03 

0.47 

1.77 

1.82 

0.05 

0.24 

4 

1 work on math 
problems during 
classtime with other 
students in my class 

2.25 

2.22 

-0.03 

0.61 

2.31 

2.26 

-0.05 

0.27 

5 

1 use physical objects 
such as cubes and 
blocks when 1 am doing 
math problems 

3.23 

3.17 

-0.06 

0.33 

3.22 

3.03 

-0.20 

0.01 
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Table 5: Mean Score Difference Between Pre- and Post-Surveys, by 

Treatment Groups (continued) 



Self -paced Cohort 
(25 classrooms, 447 students) 

Facilitated Cohort 
(21 classrooms, 386 students) 

Pre 

Post 

Post - Pre 

V 

Pre 

Post 

Post - Pre 

V 

6 

The problems we do 
in math class take 
an entire class or more 
to finish 

2.67 

2.64 

-0.03 

0.71 

2.99 

2.82 

-0.17 

0.03 

7 

Students help make 
rubrics (grading guides) 
that tell us how math 
work will be graded 

3.38 

3.35 

-0.03 

0.77 

3.58 

3.50 

-0.08 

0.33 

8 

In math class, 1 correct 
the work that other kids 
do 

3.13 

3.14 

0.01 

0.80 

3.35 

3.32 

-0.04 

0.40 

9 

We copy notes from the 
board 

2.29 

2.52 

0.23 

0.01 

2.21 

2.38 

0.17 

0.03 

10 

1 show my work with 
pictures 

3.17 

3.12 

-0.06 

0.43 

3.08 

3.11 

0.03 

0.46 

11 

We have a quiz or a test 

2.07 

2.13 

0.06 

0.34 

2.01 

2.01 

-0.01 

0.89 

12 

We use calculators in 
math class 

2.31 

2.14 

-0.18 

0.05 

2.04 

1.96 

-0.08 

0.38 

13 

We use computers in 
math class 

3.29 

3.09 

-0.20 

0.03 

3.57 

3.56 

-0.01 

0.89 

14* 

In math class, our 
teacher gives us 
worksheets that have 
many short math 
problems 

2.35 

2.44 

0.09 

0.21 

2.38 

2.27 

-0.11 

0.23 

15* 

In math class, we work 
on one or two hard 
math problems for a 
long time 

2.68 

2.78 

0.10 

0.19 

2.81 

2.71 

-0.10 

0.12 

16 

My teacher show 
us how to do math 
problems 

1.46 

1.57 

0.12 

0.09 

1.37 

1.51 

0.14 

0.10 


* Change scores differed significantly between groups with p = 0.05 and adjusted for multiple comparisons. 


Mathematics Understanding 

To examine the effect that participation in the online professional 
development course had on teachers’ knowledge of teaching mathematics, 
teachers completed an extended mathematics test that required them to 
analyze three samples of student work. Table 6 (next page) displays the 
mean consensus scores awarded for each treatment group for the pre- and 
post-course test administration. 
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Within each group, there were increases in scores for the vast majority 
of items. For the Self-Paced group, score increases were significant for five 
items, namely item 1.4, 1.5, 2.4, 3.1 and 3.4. For the Facilitated Cohort 
group, score increases were significant for four of the same items, namely 
1.4, 2.4, 3.1 and 3.4. It is interesting to note that in all cases teachers were 
ahle to provide stronger responses to the fourth prompt, which asked 
what they were left wondering about each student’s understanding and 
how they would find out. When comparing the degree of change experi- 
enced across the two groups, a significant difference was found for only 
one item, namely 3.4. While hoth groups saw significant score increases, 
the magnitude of the increase was significantly larger for the Facilitated 
Cohort group. 

Table 6: Mean scores for Pre and Post tests by Treatment Group 



Self-paced 

Learning Community 

N 

Pre 

Post 

Post- 

Pre 


N 

Pre 

Post 

Post- 

Pre 

P' 

Task 

1.1 

Identify the Content 
and Process Goals of 
this patterns problem. 

30 

1.90 

2.17 

0.27 

0.13 

23 

2.13 

2.52 

0.39 

0.10 

Task 

1.2 

What do you think 
each student 
understands about 
the mathematics of 
this problem? What is 
your evidence? 

30 

2.97 

2.93 

-0.03 

0.85 

23 

2.83 

2.87 

0.04 

0.82 

Task 

1.3 

What, if any, 
misconceptions do 
these students show? 

30 

2.53 

2.63 

0.10 

0.61 

23 

2.70 

2.65 

-0.04 

0.82 

Task 

1.4 

What are you left 
wondering about 
each student's 
understanding and 
how would you find 
out? 

30 

2.20 

2.80 

0.60 

0.01 

23 

2.26 

2.57 

0.30 

0.05 

Task 

1.5 

How would you 
deconstruct any 
misconceptions these 
students appear to 
have? 

30 

2.40 

2.97 

0.56 

0.01 

23 

2.26 

2.48 

0.22 

0.29 

Task 

2.1 

Identify the Content 
and Process Goals 
of this Functions 
problem. 

30 

1.97 

2.10 

0.13 

0.21 

23 

1.91 

2.00 

0.09 

0.49 

Task 

2.2 

What do you think 
each student 
understands about 
the mathematics of 
this problem? What is 
your evidence? 

30 

2.57 

2.87 

0.30 

0.11 

23 

2.39 

2.52 

0.13 

0.54 
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Table 6: Mean scores for Pre and Post tests by Treatment Group (continued) 



Self-paced 

Learning Community 

N 

Pre 

Post 

Post- 

Pre 


N 

Pre 

Post 

Post- 

Pre 

P' 

Task 

2.3 

What, if any, 
misconceptions do 
these students show? 

30 

2.47 

2.27 

-0.20 

0.23 

23 

2.22 

2.04 

-0.17 

0.46 

Task 

2.4 

What are you left 
wondering about 
each student's 
understanding and 
how would you find 
out? 

30 

1.77 

2.30 

0.53 

<0.01 

23 

1.70 

2.30 

0.61 

0.02 

Task 

2.5 

How would you 
deconstruct any 
misconceptions these 
students appear to 
have? 

30 

2.03 

2.23 

0.20 

0.30 

23 

1.78 

2.17 

0.39 

0.07 

Task 

3.1 

Identify the Content 
and Process Goals of 
this Number Sense 
problem. 

30 

1.93 

2.43 

0.50 

0.01 

23 

1.87 

2.43 

0.57 

<0.01 

Task 

3.2 

What do you think 
each student 
understands about 
the mathematics of 
this problem? What is 
your evidence? 

30 

2.43 

2.53 

0.10 

0.56 

23 

2.35 

2.30 

-0.04 

0.77 

Task 

3.3 

What, if any, 
misconceptions do 
these students show? 

30 

2.47 

2.40 

-0.07 

0.69 

23 

2.30 

2.43 

0.13 

0.45 

Task 

3.4 

What are you left 
wondering about 
each student's 
understanding and 
how would you find 
out? 

30 

1.90 

2.33 

0.43 

0.02 

23 

1.74 

2.74 

1.00 

<0.01 

Task 

3.5 

How would you 
deconstruct any 
misconceptions these 
students appear to 
have? 

29 

2.03 

2.24 

0.17 

0.46 

23 

2.09 

2.43 

0.35 

0.15 


' P value for dependent f test: Whether difference between post and pre scores is significant different 
from zero. 
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Discussion and Summary 

This experiment examined the effects that specific design features of 
an online professional development course have on the perceived value of 
the course and the intended learning outcomes. The specific design fea- 
tures examined included the pace of the course and the level of interac- 
tion among participants and with the course facilitator. In the Self-Paced 
version, participants worked through course readings and assignments at 
their own pace, with minimal interaction via email with the course facili- 
tator. In many respects, the Self-Paced version was similar to a guided self- 
study course that contained specific activities and writing assignments 
participants are expected to complete. In the Facilitated Cohort version, 
participants worked through the material together in weekly sessions and 
were required to interact with each other in an online discussion forum. 
In this version, the course facilitator was also proactive in communicating 
with participants and often initiated interactions with and among partici- 
pants by posting questions to the group or directly emailing individual 
participants. 

The intended learning outcomes of both courses, however, were 
identical. Specifically, The Building Algebraic Thinking in the Middle Grades 
online professional development course was intended to change teachers’ 
pedagogical beliefs, level of understanding, and practices when teaching 
algebra. To examine the effect that this OPD course had on these three out- 
comes, several data collection instruments were employed. A mathematics 
test was used to measure changes in teachers understanding of teaching 
algebra. A survey was used to measure teachers’ pedagogical beliefs. And 
the combination of a teacher survey, student survey, and instructional 
logs were used to measure teachers’ instructional practices. 

As described in greater detail above, both conditions of the course 
appear to have been effective in altering teachers’ pedagogical beliefs, 
changing their instructional practices, and increasing their understanding 
of teaching algebra. On average, teacher-directed and student-centered 
pedagogical beliefs both weakened, and instead teachers appeared to rec- 
ognize that a balanced approach to instruction was valuable. Teachers also 
appeared to decrease their value and use of worksheets, multiple-choice 
quizzes, and direct instruction, while increasing opportunities for students 
to discuss their problem solving strategies, write about mathematics, and 
engage in more extended, higher-order problem solving activities. These 
changes in practices were reported across the teacher survey, student 
survey, and the instructional logs. Finally, teachers’ understanding of 
algebra teaching also increased, particularly with respect to understanding 
how to probe student work to better understand the thinking behind stu- 
dents’ answers, even when the answers were incorrect. Each of these out- 
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comes is not surprising, particularly since teachers, on average, rated the 
quality of the course as high and reported that they learned more from 
this ten week online course than they typically learned in college or uni- 
versity courses. 

What is surprising, however, is that the effects of the course on the 
intended outcomes generally did not differ significantly between the two 
versions of the course. On the mathematics test, there was only one item 
for which a significant group difference was detected, but even for this 
item, both groups realized significant gains. With respect to pedagogical 
beliefs, only one item showed a significant difference between the two 
groups, but here again significant change occurred with both groups, but 
was larger for the Facilitated Cohort group. Finally, the instructional logs 
revealed no significant changes between the two groups, while the student 
survey showed only two items that differed significantly. These two items 
focused on the use of extended versus shorter mathematical problems to 
help develop students’ learning. Given the large number of scales and indi- 
vidual items analyzed, the fact that significant differences were detected 
for only one scale and four items provides evidence that the Self-Paced and 
the Facilitated Cohort versions of the course had approximately the same 
affect on the intended outcomes of the course. 

This finding is surprising given the emphasis in the literature on the 
importance of interactions among peer participants in online courses. 
While substantial interaction occurred in the Facilitated Cohort group, 
absolutely no interaction among participants occurred in the Self-Paced 
course. Yet, the outcomes were nearly identical. Similarly, the literature 
emphasizes the importance of online facilitation, a time consuming task 
that requires skill in managing discussions and personalities. Both the 
Facilitated Cohort classes were led by seasoned facilitators who were expe- 
rienced mathematics teachers and who have received high ratings in pre- 
vious courses. The facilitators reported investing considerable time each 
week (averaging at eleven hours) interacting with participants, and man- 
aging online discussions. While these same facilitators also led the Self- 
Paced course, the structure of those sessions did not require, or even allow, 
them to manage discussions. Although one of the facilitators did engage 
in multiple email exchanges with a few participants, the time invested in 
facilitating the Self-Paced course was substantially smaller (an average of 
three and a half hours per week), and focused primarily on providing feed- 
back on completed assignments. Thus, given the skill and time required, 
as well as the associated cost for facilitating a Facilitated Cohort course, 
the fact that both versions of the course yielded similar positive effects 
on participants suggests that a well designed online course may not need 
as much facilitation or interaction to produce the same types of learning 
outcomes. 
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These findings, however, must be placed in the context of several limi- 
tations of this study. First, the study focused on only one course delivered 
only to middle school mathematics teachers. Had a different course that 
focused on different types of teachers or different content been employed, 
the outcomes may have been different. 

Second, both versions of the course experienced considerable attrition. 
On average, 44% of the teachers who began a course did not finish the 
course or the required data collection instruments. While the literature is 
mixed with respect to retention rates for online professional development, 
the fact that 44% of teachers did not complete the course may mean that 
the course either did not meet their needs or was too challenging for them. 
Although the characteristics of teachers who dropped out of the course did 
not differ between the two versions, it is plausible that had these teachers 
persisted, different findings may have resulted. 

Third, the course employed for this study was very well designed and 
employed a high-quality text. In addition, the course lasted for ten weeks. 
While the quality of the course was not compared to other online courses, 
it is likely that many other courses are of shorter duration, employ lower 
quality reading material, or ask participants to engage in less effective 
classroom activities. For a course that is shorter in length or that employs 
materials of a lower quality, interactions among participants may provide 
valuable supplemental opportunities for learning. Thus, had this study 
employed a shorter course or a course that employed lower quality mate- 
rials and activities, different findings may have resulted. 

Finally, all of the participants in this course were volunteers. Although 
a small stipend was offered to compensate participants for the time 
required to complete the data collection instruments, the teachers who 
participated and completed the course were likely highly motivated indi- 
viduals who were sincerely interested in developing their algebraic teaching 
skills. In many cases, however, participation in professional development 
is required by a school or district. In such cases, some teachers may be 
less motivated and engaged in the learning. If presented with a self-paced 
version of the course, some of these teachers might be unmotivated to 
complete the reading and activities, and may make a minimal effort when 
completing assignments. The resulting effects of the course might then be 
smaller. 

Despite these limitations, this study provides preliminary evidence that 
an online course designed as a self-study may be equally effective as one 
that moves an interactive Facilitated Cohort together through a series of 
readings, activities, and assignments. Future studies may wish to explore 
this issue using different courses, content areas, and approaches to partici- 
pant recruitment. In addition, it would be informative to examine whether 
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even more extreme differences in the level of interaction and facilitation 
affected the outcomes of the course. What is clear here, though, is that an 
online professional development course can have very positive effects on 
teachers’ knowledge, pedagogical beliefs, and instructional practices, and 
that the magnitude of these effects may not always differ between courses 
that are designed to occur as Self-Paced or as Facilitated Cohort learning 
opportunities. 
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